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ABSTRACT 

To replicate and extend Grey's (1981, 1985) studies 
involving junior-high school students, "How Is Your Logic?" (a 
26-item, Piaget ian-baj ed, group-administered written test of 
cognitive development) was given to 553 subjects, 10 through 48 years 
of age. Each item of the test measured either a specific concrete 
operation or a specific formal operation. Image analyses of the 
inter-item correlation matrix of the formal operations items alone 
completely replicated the results of a previous study, and image 
analyses of the inter-item correlation matrix of all the items 
together partially replicated the results of the same previous study. 
These results are interpreted as providing support for the contention 
that concrete operations and formal operations are defined by 
separate but interrelated logical processes. (Author/RH) 
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Abstract 

A twenty-six item, Piagetian-based, group-administered written 
test of cognitive development, with each item measuring a specific 
concrete operation or a specific formal operation, was given to 
553 subjects, ages 10-48. Image analyses of the inter-item 
correlation matrix of the formal operations items alone completely 
replicated the results of a previous study and image analyses of 
the inter-item correlation matrix of all the items together 
partially replicated the results of the same previous study. These 
results are interpreted as providing support for the contention that 
concrete operations and formal operations are defined by separate 
but interrelated logical processes. 
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The Structure of Operational Thought? A Replication and Extension 

Gray (1981, 1985) suggested that the relations among various 
individual concrete operations and formal operations do conform to 
Piaget's idea of structures d y ensemble. That is, intrastage- 
intraoperation relations are stronger than intrastage-interoperation 
relations and intrastage-interoperation relations are stronger 
than interstage-interoperation relations* He hypothesized that 
such relations should be manifest in a factor analysis where each 
variable specifically measures one concrete operation or one formal 
operation. Intrastage-intraoperation relations should be represented 
by clearly defined factors where the underlining structure of each 
factor corrsponds to one concrete operation or one formal operation. 
Intrastage-interoperation relations and interstage-interoperation 
relations should be represented by the correlations among the various 
factors. 

In presenting his argument, Gray critiqued a number of studies 
that focused on the concept of structures d T ensemble. In many 
studies, the variables did not clearly measure one concrete operation 
or one formal operation, thus, making it difficult to determine 
what was measured and eliminating any reasonable conclusion 
about the operations represented by the variables. Most factor 
analytic-based studies included relatively weak factor analytic 
techniques (e.g., principal components analysis) and/or consistently 
used an orthogonal transformation of the initial factor matrix which 
makes it almost impossible to investigate the relations among any 
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of the factors because an orthogonal transformaion automatically 
assumes that there is no relationship among the factors. 

In contrast to previous studies, Gray (1981, 1985) used a 
robust faccor anaiyatic technique (Harris image analysis, Harris, 
1962) and Hofmann's (1978d) orthotran transformation procedure as 
well as variables that clearly represented one concrete operation 
or one formal operation. Using twenty-four items from two forms 
of a group-administered, Piagetian-based test of cognitive development, 
he obtained four clear formal operations factors: Make Correct 
Exclusions/Deny Incorrect Inclusions, Probability/Proportional 
Reasoning, Make a Correct Inclusion, and a generalized Combinatorial 
Thought Factor. These four formal operations factors were found 
both when the formal operations items were analyzed alone or when 
they were analyzed with the concrete operations items. When the 
concrete operations items were analzyed alone there was a 
Multiplication of Classes Factor and an Addition of Asymmetrical 
Relations Factor. When all items were analzyed together the 
Addition of Asymmetrical Relations Factor and three formal operations 
factors remained separate. The formal operations Combinatorial Thought 
Factor was differentiated into two separate factors, one that was 
mixed with the concrete operations Multiplication of Classes Factor 
and a separate Combinatorial Thought Factor. These results were 
interpreted as providing support for the idea that individual 
concrete operations and individual formal operations are isolatable 
but related to each other. Gra> s data were Used on 577 junior 
high school students which included the ages during which it is 
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hypothesized (Inhelder & Piaget, 1955/1958, 1959/1969; Piaget, 1947/1966, 
1953/1957) that concrete operations evolve into formal operations. 
However, empirically (see Blasi & Hoeffel, 1974; Keating, 1980; Neimark, 
1975, 1979 1982, for reviews), concrete operations evolve into 
formal operations over a much greater age span than that originally 
suggested by Piaget. The present study replicated Gray's and 
extended it by including subjects whose ages encompass the ages 
during which concrete operations empirically evolve (if they ever 
do evolve) into formal operations. 

Method 

Subjects 

The original sample included 746 subjects involved in a study 
which focused on the relations between moral reasoning and 
operational thought. Mean age was 16.474 years (s - 5.195) 
ranging from 9.481 to 48.463 with 55.5% males and 44.5% females. 
Subjects were from a suburb of a medium-size midwestern city and 
two introductory required logic classes in the state supported 
university within the city. Because of scheduling problems, 
approximately 200 subjects did not complete both forms of the logic 
test. Consequently, 553 subjects (55.3% male, 44.7% female) 
completed both measures of operational thought and are included in 
the present study. Mean age of the 553 subjects was 16.662 years 
(s « 4.970) ranging from 10.949 to 48.463. 
Procedure 

The procedure and analyses follow exactly those reported by 
Gray (1981, 1985). Within a two week period, subjects were given 
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Form A and Form B, respectively, of How Is Your Logic? (Gray, 1976a, 
1976b), a Piagetian-based, group-administered written test of 
cognitive development. Each form includes thirteen constructed 
response items with each item measuring a specific concrete 
operation or a specific formal operation. Across both forms, 
three concrete operations and four formal operations are assessed 
by a minimum of two items per operation. Responses to each item 
are evaluated according to the reasoning manifested in the responses. 
For example, a response to a formal operations item can be scored 
as p reoperations, preope rat ions-concrete I, concrete I, concrete 
II, concrete II-formal I, formal I, or formal II, depending upon 
whether the item is a beginning formal operations item or a consolidated 
formal operations item. Similarly, responses to concrete operations 
items are scored as preoperations, preoperations-concrete I, or 
cmcrete I. AH items are scored conservatively. That is, if a 
subject's response clearly indicates a concrete II level of thinking 
and only partially indites a formal I level of thinking the 
response is scored at a concrete II level. This approach to 
scoring responses is in line with the Genevan approach which 
requires subjects to clearly demonstrate a specific operation 
before it can be indicated that an individual has mastered the 
operation. 
Analyses 

Scorers were doctoral students in educational psychology. All 
scorers were provided instruction in the scoring by the author who 
designed the scoring procedures. Each scorer was required to first 
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score 25-30 tests and the scores on the individual items were then 
matched with the author's scores for the items. Any discrepancies 
in scores were discussed and appropriate clarification of the 
scoring criteria were made. Subsequently, the tests that were used 
as training devices were returned to the subject pool and were 
rescored along with the remaining tests. Percent agreement for 
each item, where raters 1 scores were compared with the score given 
by the author, ranged from approximately 85% to 100Z with only five 
items having an agreement score less than 100%. (All five items 
required scorers to evaluate sentences written by the subjects.) 
Coefficient alpha estimates of internal consistency were very good 
for the concrete and formal items together (.82) and the formal 
items alone (.83). The alpha for the concrete items alone was .57 
and was a direct result of a ceiling effect on the items. As in 
the original Gray (1981, 1985) study, the Multiplication of 
Relations item from each form was eliminated from any analysis. 

Because there is concern that correlations generated from 
subjects encompassing such a wide range of ages may artificially 
inflate the correlations (Carroll, Kohlberg, & DeVries, 1984; 
Humphreys & Parsons, 1979), interitem partial correlation matrices, 
with age partialed-out, were calculated. Next, each matrix of 
partial correlations was subjected to a series of Harris image 
analyses with the normalized initial factor matrices transformed 
via Hofmann's (I978d) orthotran. Average Kaiser-Meyer-Olkin 
measures of sampling adequacy (Kaiser, 1970) were excellent for the 
formal operations items and concrete operations items together 
(.80) and the formal operations items alone (.80), and good for 
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the concrete operations items alone (,68). (As with the alpha 
coefficients, the relatively low average measure of sampling 
adequacy was the result of b ceiling effect on the concrete items,) 
A three part approach was used to determine the number of factors 
to extract from each of the partial correlation matrices. The 
approach that was used is described in Gray (1981, 1985) and 
involved the use of Hofmann's (1977, 1978a) index of variable 
complexity, Cattell's (1966) scree test, and the psychological 
interpretability of the various solutions that were generated from 
each data set. For the formal operations items and concrete 
operations items together, the scree test and the psychological 
interpretabiity of the various solutions suggested there were four 
factors whereas the complexity criterion suggested there were three 
factors. Thus, four factors were extracted from the total set of 
items. Four factors were extracted from the formal operations 
items alone as bach criterion suggested there were four factors in 
the formal operations data. Because the concrete operations items 
did not define one or more separate factors in the solutions of the 
concrete items and formal items together no factors were extracted 
from the concrete items alone, 

,1 



Results J 



The primary pattern solution for the formal items and concrete 
items together is presented in Table 1 and the correlations, 
multiple correlations, and partiaL correlations among the factors 
presented in Table 1 are presented in Table 2. The average variable 
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Insett Tables 1 and Table 2 about here 



complexity is quite low at 1.209 and the simple structure solution 
of the factors is excellent with only Factor III not defined by 
one clear operation. Factor I represents the formal operation of 
Probability/Proportional Reasoning, Factor II the formal operation 
of Make Correct Exclusions/Deny Incorrect Inclusions, and Factor 
IV the formal operation of Make a Correct Inclusion. These factors 
are comparable to three of the six factors found in the analogous 
solution in Gray (1981, 1985). Specifically, Factors I, II, and 
IV are identical to Factors III, II, and V, respectively, reported 
by Gray (1981, 1985). Factor III is an integrated formal operations- 
concrete operations factor and is analogous to Gray's (1981, 
1985) Factor I. The difference between the two solutions is 
that Gray's (1981, 1985) concrete operations and formal operations 
factor was basically a concrete operations Multiplication of Classes 
Factor with substantial but smaller coefficients with the beginning 
formal operations combination items. In the present study, the 
major coefficients on the integrated factor are with the formal 
operations combinatorial thought items and smaller but acceptable 
coefficients with concrete operations multiplication of classes 
items. The addition of asymmetrical relations items defined a 
separate factor in Gray (1981, 1985); in the present study, only 
one of the items had a coefficient greater than .300, and that 
was with the integrated formal operations-concrete operations factor. 
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The interfactor correlations and partial correlations reported in 
Table 2 are reasonable for such measures (Hofmann, 1978c, pp. 9-10) 
and, as reported by Gray (1981, 1985), the largest partial correlation 
is between the beginning formal operation of Make a Correct 
Inclusion and the consolidated formal operation of Make Correct 
Exclusions /Deny Incorrect Inclusions* In essence, the present 
data clearly replicated three operations-based factors found 
in an earlier study the fourth factor is analogous to another 
factor found in the earlier study. 

Table 3 presents the primary pattern solution for the formal 
items alone and Table 4 presents the correlations, multiple 
correlations, and partial correlations among the primary factors 
represented in Table 3. The factors represented in Table 3 are an 



Insert Tables 3 and 4 about here 



elegant solution. All of the factors are clearly defined by one 
of the four formal operations measured by the 16 items: Make 
Correct Exclusions/Deny Incorrect Inclusions, Probability/ 
Proportional Reasoning, Combinatorial Thought, and Make a Correct 
Inclusion, Factors I, II, III, and IV, respectively. The average 
variable complexity of 1.035 is outstanding and is slightly 
better than -he 1.053 reported by Gray (1981, 1985) for the formal 
items. The values reported in Table 4 suggest reasonable relationships 
among the factors. The pattern of coefficients for the partial 
correlations exactly duplicates that reported by Gray for his 
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solution of the iocmal items. The operation^ necessary for success on 
the items defning each of the four factors are the same operations 
necessary for success on four of the Inheider probiems (Inhelder & 
Piaget, 1955/1958). Make Correct Exclusions/Deny Incorrect Inclusions 
(Factor I) and Make a Correct Inclusion (Factor IV) are the operations 
necessary for success at the consolidated formal operations substage 
and the beginning formal operations substage, respectively, on the 
pendulum problem; Probability/Proportional Reasoning (Factor II) ts 
analogous to the operations necessary for success on the equilibrium 
in the balance problems; and Combinatorial Thought /Factor III) is 
analogous to the operations necessary for success on the chemical 
problem and the permutation problem (Piaget, & Inhelder, 1951/1975). 
Thus, correlations among the various traditional Piagetian tasks 
may be interpreted as analogous to the interf actor correlations 
reported in Table 4. The separate correlations between Combinatorial 
Thought and the two exclusion-based factors (Make a Correct Inclusion 
& Make Correct Exclusions/Deny Incorrect Inclusions) are similar to 
or greater than one reported by Kuhn t Langer, Kohlberg, and Haan 
(1977/, and smalLer than one reported by Shayer (1979) for the 
relation of the combinations of colored and colorless chemical 
bodies problem with the pendulum problem. Similarly, the correlations 
between the two exclusion-based factors (I, IV) and the Probability/ 
Proportional Reasoning factor (II) as well as the correlation between 
Combinatorial Thought (III) and Probability/Proportional Reasoning 
(II) are comparable but smaller than those reported by Shayer (1979) 
tor the relation of the pendulum problem with the balance problem and 
the chemicals problem with the balance problem, respectively. 

12 
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Although these cross-study relations are suggestive, the differing 
levels at which the relations are occurring must be considered 
(i.e., zero-order correlations vs. interfactor correlations). 

Also, as reported by Gray (1981, 1'85), when the concrete 
items are removed from the analysis, the coefficients defining the 
Combinatorial Thought factor (III) are all larger than when the 
concrete items are included in the analysis. 

Because none of the concrete operations items defined a separate 
factor when the formal items and concrete items were analyzed 
together, and because of a "ceiling effect" on the concrete items 
which dramatically reduced the variance among the concrete items, 
it was decided that there was no necessity to analyze them separately. 
(R. J. Hofmann, Personal Communication, April 23, 1985). 

Discussion 

Three important points can be made about the results of the 
present study. First, and probably most important, is the exact 
replication of the results on the formal items and a partial replication 
of the results on the formal items and concrete items together that 
were reported by Gray (1981, 1985). He presented a detailed discussion 
of the theoretical and methodological implications of his results 
stressing that individual factors represent intrastage-intraoperat* 
relations, and interfactor correlations and partial correlations 
represent intrastage-interoperation relations as well as interstage- 
interoperation relations. It was his contention that such relationships 
as defined by his factor analytic results clearly supported Piaget's 
ideas regarding the structures d f ensemble of formal operations and 
concrete operations. As noted previously, however, his sample was 
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restricted to junior high schooi students and, thus, did not completely 
cover the age span during which one would expect concrete operations to 
evolve into formal operations. Such a restriction of subjects' 
ages may have accounted for his results. If Gray's results were a 
function of the ages of the subjects then the greater age range 
included in the present study should have produced a factor 
structure different than the structure found in the previous study. 
However, the theoretically-based factor structure reported by 
Gray, especially with the formal operations items alrne, was replicated 
exactly. This replication with an extended age range supports 
Piaget's ideas regarding the relationships among different operations 
that define formal operations. 

A second major result revolves around the relations among the 
formal items and concrete items. In several factor analytic-based 
studies where concrete operations and formal operations have been 
investigated together, the concrete operations variables and the 
formal operations variables tend to define or are interpreted as 
defining one general concrete operations factor and one general 
formal operations factor (Guerin, 1975; Lawson, 1976, 1978; 
Lawson & Nordland, 1976; Lawson & Renner, 1974, 1975). Theoretically, 
these results are seen as supporting the separate wholistic 
quality (structures d'ensemble ) of each operational stage. However, 
Piaget continually stressed that as concrete operations evolve 
into formal operations, concrete operations become the conter 
for formal operations and there is a hierachicr.l relationship that 
is established between concrete operations and formal operations. 
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That is, concrete operations are incorporated within formal 
operations both as content for formai operations and as toois for 
use by formal operations. Such a perspective suggests chat as 
formal operations begin to dominate thought, the impact of concrete 
operations lessen. Thus, a combined formal operations and concrete 
operations factor, where formal operations items define the factor 
and concrete operations items have substantial but smailer 
coefficients with the factor, may be interpreted as supporting such 
a hierarchicai deveioment, especialiy if the subject's ages encompass 
the ages during which concrete operations evoive into formai operations 
(R. J. Hofmann, personai communication, Aprii 23, 1985). 
A comparison of the integrated formai operations-concrete 
operations factor in Tabie 1 and Gray's (1931, 1985) mixed concrete 
operations and formai operations factor may be interpreted as 
supporting this position. In the present study, the formai operations 
combinatorial thought items have iarger coefficients with the integrated 
formal operations-concrete operations factor than they do with the mixed 
concrete operations and formai oeprations factor reported by Gray 
(1981, 1985). Also, the coefficients of the concrete operations 
items with the integrated formal operations-concrete operations 
factor in Table 1 are smaller than with the mixed concrete operations 
and formai operations factor reported by Gray (1981, 1985). In fact, 
three out of four seriation items are so small that they are no longer 
considered as an item that defines a factor. Similar results with formal 
operations or concrete operations dominating a mixed factor have been 
reported, but not interpreted as such, by Lawson and Nordland (1976), 
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Lawson and Renner (1974, 1975) f and Lee (1971) , suggesting support 
for Piaget's ideas regarding the changing relationships between 
concrete operations and formal operations across the ages during 
which concrete operations evolve into formal operations. Obviously, 
only a long-term longitudinal study can adequately provide data to 
address this issue. 

A final important result is that the content of problems 
used to assess operational thought is not as important as the 
operations needed to correctly solve the problems. Clearly, this 
conclusion was again supported as the items that were used were 
not traditional Piagetian tasks. They were written items that 
required the logic of formal operations or concrete operations to 
successfully solve the problems but the items did not include any 
of the content of traditional Inhelc'er or Piaget tasks. For 
example, the various exclusion items (Make Correct Exclusions/Deny 
Incorrect Inclusion and Make a Correct Inclusion) incorporated the 
logic required to solve the Inhelder pendulum problem. However, 
none of the content of those written exclusion items included any 
reference to an understanding of the functioning of a pendulum. 
These results with non-traditional tasks support beliefs by 
Roberge and Flexer (1979), Gray (1978a, 1978b), Neimark (1975) and 
others who have indicated that different types of problems that 
require operational thought to be successful need to be developed. 
The items on How Is Your Logic? are such a set of problems. 
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Footnotes 

Coefficient alpha estimates of internal consistency, were 
computed using SPSS* Release 2.0 for IBM VM/CMS. The Harris imag< 
analyses and orthotran transformations were computed using MUFACT 
(Uofmann, 1978b). 
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Table 1 

Harris Image Analysis Primary Pattern Solution for Formal Items and Concrete 
Items 



Factor Variable 

Form & 

Operation Item I II III IV Complexity 13 



Probability 
Judgment-FI 

-FII 
Explanation 

Exclusion-Correct 
Judgment 

Explanation 

Combinatorial Thought 
Combinations 

Permutations 

Multiplication of Classe 
One-to-One 



A12 _700 -0 1 9 

Bll J78 -077 

A13 841_ 060 

B12 761 000 

A9 005 _721_ 

B3 -084 _708 

A10 018 723 

B4 -034 _73_7 

A$ -003 -095 

B6 015 -074 

A6 189 -048 

B7 - 197 -121 

A7 -035 -054 

B2 -039 152 



003 -003 1.001 

-049 -073 1.045 

-028 -042 1.017 

020 064 1;015 

-046 -028 1.011 

022 074 1.052 

-010 028 1.005 

044 037 1.107 

717 041 1.042 

671 009 1.025 

600 157 1.358 

543 211 1.705 

512 062 1.062 

534 -11$ 1.288 



(table continues) 
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Factor Variable 

Form & 



Operation item I II III IV Complexity 15 



Many-to-One 


All 


089 


155 


465 


-222 


1.776 




B13 


-024 


084 


467 


-093 


1. 152 


Seriation 














Increasing Series 


Al 


-097 


001 


292 


-007 


1.219 




Bl 


-010 


-060 


254 


044 


1.178 


Decreasing Series 


A4 


-000 


107 


_364 


-210 


1.800 




B5 


007 


020 


152 


-196 


1.914 


Indus ion-Correct 














Judgment 


A2 


076 


157 


-007 


591 


1.176 




B9 


-088 


-024 


012 


865 


1.023 


Explanation 


A3 


072 


093 


053 


622 


1.088 




BIO 


-021 


119 


024 


772 


1.051 



Note * Probability -« Probability/Proportional Reasoning operation; FI - 
beginning formal operations; FI* « consolidated formal operations; 
Exclusion-Correct - Make Correct Exclusions/Deny Incorrect Inclusions 
aspect of the Exclusion operation; One-to-One - One-to-One Correspondence 
aspect of the Multiplication of Classes operation; Many-to-One « 
Many-to-One Correspondence aspect of the Multiplication of Classes 

(table continues) 
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operation; Seriation * Addition of Asymmetrical Relations operation; 

Inclusion-Correct - Make a Correct Inclusion aspect of the Exclusion 

operation, 
a 

Decimal points have been eliminated and coefficients greater than 
300 are in italics, 

b Average variable complexity - 1.209. 
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Table 2 

Correlations, Multiple Correlations, and Partial 
Correlations Among the Primary Factors for 
Formal Items and Concrete Items 



Factor I II m IV 



I 


.343 


.174 


.247 


.300 


II 


.032 


.433 


.276 


.401 


III 


.164 


.168 


.380 


.304 


IV 


.219 


.331 


.176 


.489 



Note* Correlations in upper right, multiple 
correlations in main diagonal, and partial 
correlations in lower left* 
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Table 3 



Harris Image Analysis 


Primary Pattern 


Solution 


for For 


mal Items 










Factor 




Variable 




Form & 












Operation 


Item 


I 


II 


III 


IV 


Complexity' 3 


Exclusion-Correct 














Judgment 


A9 


689 


012 


-118 


017 


1.061 




B3 


768 


-095 


050 


-019 


1.040 


Exp lanation 


A10 


751 


018 


-025 


-015 


1. 004 




B4 


808 


-036 


046 


-057 


1.021 


Probability 














Judgment-FI 


A12 


-033 


708 


-on 


022 


1.005 


-FII 


Bll 


-092 


802 


-059 


-067 


1.052 


Explanation 


A13 


051 


834 


006 


-059 


1.018 




B12 


000 


761 


048 


034 


1.012 


Combinatorial Thought 














Combinations 


A5 


-Oil 


-084 


805 


-079 


1.041 




B6 


000 


-070 


772 


-012 


1.052 


Permutations 


A6 


-001 


100 


693 


069 


1.062 




B7 


-059 


097 


714 


076 


1.075 


Inclusion-Correct 














Judgment 


A2 


051 


056 


-065 


702 


1.040 




B9 


-077 


-112 


006 


892 


1.047 



(table continues) 
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Factor Variable 

Form & 



Operation item I II III IV Complexity 1 * 



Explanation A3 -004 048 -001 J73 1.009 

B10 078 -043 005 792 1.025 



Note. Exclusion-Correct = Make Correct Exclusions/Deny Incorrect Inclusions 
aspect of the Exclusion operation; Probability » Probability/Proportional 
Reasoning operation; FI = beginning formal operations; FII * consolidated 
formal operations; Inclusion-Correct = Make a Correct Inclusion aspect of 
the Exclusion operation. 

Decimal points have been eliminated and coefficients greater than 
300 are in italics. 
b Average variable complexity * 1.035 
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Table 4 



Correlations, Multiple Correlations, and Partial 



Correlations 


Among the 


Primary 


Factors 


for 


Formal T tems 










Factor 


I 


II 


III 


IV 


I 


.518 


.207 


.301 


.501 


II 


.011 


.406 


.333 


.346 


III 


.117 


.223 


.464 


.402 


IV 


.429 


.217 


.240 


.597 



Note. Correlations in upper right, multiple 
correlations in main diagonal, and partial 
correlations in lower left. 
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